{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-07-31T02:22:14.814998Z",
     "iopub.status.busy": "2022-07-31T02:22:14.814437Z",
     "iopub.status.idle": "2022-07-31T02:22:16.705021Z",
     "shell.execute_reply": "2022-07-31T02:22:16.704276Z"
    },
    "init_cell": true,
    "origin_pos": 2,
    "slideshow": {
     "slide_type": "notes"
    },
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import random\n",
    "import torch\n",
    "import matplotlib.pyplot as plt\n",
    "from IPython.core.interactiveshell import InteractiveShell\n",
    "InteractiveShell.ast_node_interactivity = \"all\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "<div class=\"jumbotron\">\n",
    "    <h1 class=\"display-1\">线性回归的从零开始实现</h1>\n",
    "    <hr class=\"my-4\">\n",
    "    <p>主讲：李岩</p>\n",
    "    <p>管理学院</p>\n",
    "    <p>liyan@cumtb.edu.cn</p>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## 线形回归的基本元素"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "### 线形模型"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- 假设数据集有$d$个特征，对于第$i$个样本，特征为集合$\\{x_{1}^{(i)},x_{2}^{(i)},\\cdots,x_{d}^{(i)}\\}$，则该样本的预测结果$\\hat{y}^{(i)}$可以表示为\n",
    "\n",
    "$$\n",
    "\\hat{y}^{(i)}=w_1x_{1}^{(i)}+w_2x_{2}^{(i)}+\\cdots+w_dx_{d}^{(i)}+b\n",
    "\\label{eq:lineareqn}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- 第$i$个样本的所有特征用向量$\\boldsymbol{x}_i\\in\\mathbb{R}^d$表示，所有权重用向量$\\boldsymbol{w}\\in \\mathbb{R}^d$表示，则式\\eqref{eq:lineareqn}可用向量表示为\n",
    "\n",
    "$$\n",
    "\\hat{y}^{(i)}=\\boldsymbol{w}^T\\boldsymbol{x}^{(i)}+b\n",
    "\\label{eq:linearvec}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- 全部数据集（含有$n$个样本）的特征用矩阵$\\mathbf{X}\\in\\mathbb{R}^{n\\times d}$表示，所有样本的预测值用向量$\\hat{\\boldsymbol{y}}\\in\\mathbb{R}^n$表示，则线性模型可表示为\n",
    "\n",
    "$$\n",
    "\\hat{\\boldsymbol{y}}=\\mathbf{X}\\boldsymbol{w}+b \n",
    "\\label{eq:linearmatrix}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- 线性模型的目标是求解**模型参数**（model parameters）\n",
    "    - $\\boldsymbol{w}$\n",
    "    - $b$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### 损失函数"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "\\begin{definition}\\label{def:lossfun}\n",
    "损失函数（loss function）：量化目标的实际值与预测值之间的差距。\n",
    "\\end{definition}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- 通常选择**非负数**作为损失，且**数值越小表示损失越小**，完美预测时的损失为0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- 对于线形模型\\eqref{eq:linearvec}，真实值为$y^{(i)}$，预测值为$\\hat{y}^{(i)}$，可以用**平方误差**函数作为损失函数\n",
    "\n",
    "$$l^{(i)}(\\boldsymbol{w}, b) = \\frac{1}{2} \\left(\\hat{y}^{(i)} - y^{(i)}\\right)^2.$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- 为度量模型在整个数据集上的质量，需计算在训练集$n$个样本上的损失均值，即\n",
    "\n",
    "$$L(\\boldsymbol{w}, b) =\\frac{1}{n}\\sum_{i=1}^n l^{(i)}(\\boldsymbol{w}, b) =\\frac{1}{n} \\sum_{i=1}^n \\frac{1}{2}\\left(\\boldsymbol{w}^\\top \\boldsymbol{x}^{(i)} + b - y^{(i)}\\right)^2.$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- 线形模型\\eqref{eq:linearmatrix}的优化目标为\n",
    "\n",
    "$$\\boldsymbol{w}^*, b^* = \\operatorname*{argmin}_{\\boldsymbol{w}, b}\\  L(\\boldsymbol{w}, b).\n",
    "\\label{eq:linearobj}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### 随机梯度下降，求解式\\eqref{eq:linearobj}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "\\begin{definition}\\label{def:gradientdescent}\n",
    "**梯度下降**（gradient descent）：通过不断地在损失函数递减的方向上更新参数来降低误差。\n",
    "\\end{definition}\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- 计算损失函数（数据集中所有样本的损失均值）关于模型参数的导数（在这里也可以称为梯度）\n",
    "- 但需遍历整个数据集，执行速度慢"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "\\begin{definition}\\label{def:minibatchdescent}\n",
    "**小批量随机梯度下降**（minibatch stochastic gradient descent）：在每次需要计算更新的时候随机抽取一小批样本计算损失函数的梯度，更新参数。\n",
    "\\end{definition}\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- 首先，随机抽样一个小批量$\\mathcal{B}$，它是由**固定数量**的训练样本组成的。\n",
    "- 然后，计算小批量的平均损失关于模型参数的导数（也可以称为梯度）。\n",
    "- 最后，将梯度乘以一个预先确定的正数$\\eta$，并从当前参数的值中减掉。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "$$(\\boldsymbol{w},b) \\leftarrow (\\boldsymbol{w},b) - \\frac{\\eta}{|\\mathcal{B}|} \\sum_{i \\in \\mathcal{B}} \\partial_{(\\boldsymbol{w},b)} l^{(i)}(\\boldsymbol{w},b).$$\n",
    "\n",
    "上式中，$|\\mathcal{B}|$表示每个小批量中的样本数，称为**批量大小**（batch size），$\\eta$表示**学习率**（learning rate）。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "\\begin{definition}\\label{def:hyperpara}\n",
    "**超参数**（hyperparameter）：可以调整但不在训练过程中更新的参数。\n",
    "\\end{definition}\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "\\begin{definition}\\label{def:tuning}\n",
    "**调参**（hyperparameter tuning）：选择超参数的过程。\n",
    "\\end{definition}\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### 线性模型的神经网络表达"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- 线性回归可表示成一个**单层神经网络**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "<center><img src=\"../img/3_linear_network/singleneuron.svg\" width=80%></center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "\\begin{definition}\\label{def:denseLayer}\n",
    "**全连接层**（fully-connected layer）：每个输入都与每个输出相连，或称为**稠密层**（dense layer）。\n",
    "\\end{definition}\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## 生成样本数据"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "根据带有噪声的线性模型构造一个人造数据集。使用线性模型参数$\\boldsymbol{w} = [2, -3.4]^\\top$、$b = 4.2$\n",
    "和噪声项$\\varepsilon$生成数据集及其标签：\n",
    "\n",
    "$$\\boldsymbol{y}= \\mathbf{X} \\boldsymbol{w} + b + \\boldsymbol{\\varepsilon}$$\n",
    "\n",
    "其中，$\\varepsilon$服从均值为0的正态分布，不妨将其标准差设为0.01。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-07-31T02:22:16.717009Z",
     "iopub.status.busy": "2022-07-31T02:22:16.716673Z",
     "iopub.status.idle": "2022-07-31T02:22:16.720862Z",
     "shell.execute_reply": "2022-07-31T02:22:16.720150Z"
    },
    "origin_pos": 7,
    "slideshow": {
     "slide_type": "fragment"
    },
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [],
   "source": [
    "def synthetic_data(w, b, num_examples):  \n",
    "    \"\"\"生成y=Xw+b+噪声\"\"\"\n",
    "    X = torch.normal(0, 1, (num_examples, len(w)))\n",
    "    y = torch.matmul(X, w) + b\n",
    "    y += torch.normal(0, 0.01, y.shape) # y.shape: torch.Size([1000,1])\n",
    "    return X, y # y.shape: torch.Size([1000, 1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "```python\n",
    "torch.normal(mean, std, size, *, out=None)\n",
    "```\n",
    "- 生成正态分布的张量\n",
    "- `size`：由整数构成的元祖，定义生成的张量的形状"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "true_w = torch.tensor([2, -3.4]).reshape((-1,1))  #变成列向量\n",
    "true_b = 4.2\n",
    "features, labels = synthetic_data(true_w, true_b, 1000)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- `features`中的每一行都包含一个二维数据样本，`labels`中的每一行都包含一维标签值（一个标量）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-07-31T02:22:16.725012Z",
     "iopub.status.busy": "2022-07-31T02:22:16.724588Z",
     "iopub.status.idle": "2022-07-31T02:22:16.730034Z",
     "shell.execute_reply": "2022-07-31T02:22:16.729030Z"
    },
    "origin_pos": 9,
    "slideshow": {
     "slide_type": "fragment"
    },
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "features: tensor([1.3937, 0.4697]) \n",
      "label: tensor([5.3929])\n"
     ]
    }
   ],
   "source": [
    "print(f'features: {features[0]} \\nlabel: {labels[0]}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "- 绘制散点图观察特征与标签之间的关系"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-07-31T02:22:16.734892Z",
     "iopub.status.busy": "2022-07-31T02:22:16.734207Z",
     "iopub.status.idle": "2022-07-31T02:22:16.948832Z",
     "shell.execute_reply": "2022-07-31T02:22:16.948171Z"
    },
    "origin_pos": 11,
    "slideshow": {
     "slide_type": "fragment"
    },
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.collections.PathCollection at 0x7f60b4647208>"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXIAAAD4CAYAAADxeG0DAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAABHuElEQVR4nO29f3AUZ5rn+X2rpLIEEgIZIX5Y0IBAbtlcg5sw9hqMCcPS23F79sTGRaxxj/fMRnuY9Zx7J659s7ET4e3rjY3YXXw710x4mu2+wTPuBjt2YrbbHRPe0YAPG8zY0BjjBlcjEDBCSOgHCAkJJEqqeu+Pqjf1vm++mZVZlaWqrHo+EQ6jqqzMN7Mqv++Tz/v8YJxzEARBEOElUuwBEARBEPlBQk4QBBFySMgJgiBCDgk5QRBEyCEhJwiCCDlVxTjowoUL+de+9rViHJogCCK0fP755zc5503660UR8q997Ws4ffp0MQ5NEAQRWhhj3abXybVCEAQRckjICYIgQg4JOUEQRMghIScIggg5JOQEQRAhh4ScIAgi5BQl/JAgCCJsHI4P4PilIWxZ04Qd7c3FHo4CWeQEQRBZOBwfwGvvfoF3Pu3Ga+9+gcPxgWIPSYGEnCAIIgvHLw1hYioJAJiYSuL4paEij0iFhJwgCCILW9Y0obY6CgCorY5iyxpblnxRIR85QRBEFna0N2PfCxtK1kdOQk4QBOGBHe3NJSfgAnKtEARBhBwScoIgiJBDQk4QBBFySMgJgiBCDgk5QRBEyPEs5IyxA4yxQcbYeem1HzDGehljZzP/fbswwyQIgiCc8GOR/wWAbxle/xPO+frMfx8EMyyCIAjCK56FnHN+DMBwAcdCEARB5EAQPvI/YIz9JuN6WeC0EWPsFcbYacbY6aGh0qpTQBAEEWbyFfIfA1gNYD2AGwD+b6cNOec/4Zxv5JxvbGoqrToFBEEQYSYvIeecD3DOk5zzFICfAng8mGERBFHKHI4P4I33z5dcOddKJS8hZ4wtkf78HQDnnbYlCKI8KPXa3JWIn/DDdwF8CqCNMXadMfYvAfxnxtg5xthvAGwD8IcFGidBECVCqdfmrkQ8Vz/knL9gePnPAxwLQRAhYMuaJvzV6euYmEqWZG3uSoTK2BJEHpRyH8dCUeq1uSsREnKCyBHhK56YSuKvTl/Hvhc2VIyolXJt7kqEaq0QRI6Qr5goFUjICSIHDscH0DN8D7Fo+hYiXzFRTMi1QhA+kV0qsWgE29qasGvTCnI1EEWDhJwgfCK7VBLJFFoa59hEvBIXQYtJpV9vcq0QhE+2rGlCbXUUgNmlQgkzzhQiI5SuNwk5QfhGhN+99OQKY6QKLYKaKZTg0vUmIScqgEJYgTvam/HD5x41PsZns9grlUIJLl1v8pETZU4xYr0pYcZMoTJC6XqTkBNljskKnI0bnRJm7BRScCv9epOQE2UN1QUpLSpdcAsFCTlR1vixAis9hI0ILyTkRNkiC/MPn3s067ZhrZsS5AREk1k4oagVoizxG+oW1hC2XEP6TJE8FI8dXkjIibLErzCHNYQtlwlob0cn9vz8c5tgh3UyI0jIiTLFrzBnS/IpVfye5+H4APZ/1IVkigNQBTusk1mYKFSvU8Y5D3SHXti4cSM/ffr0rB+X8E+YfaZhHrvAyznI2wBw3f6N98/jnU+7rb+jEYb93/mmtW05XLNSRV6Hqa2O5mQwMMY+55xvtL1OQk44EcQPj8gdv9ffbXsh0PU11TjwyVVMTCURZcCeZ1rx+s622TqlglPKE5E+ib705Iqsi/A6TkJOUSuEI8VKpiHS+L3+h052G7fXBX735pUYm5wqSbHLh1KPPCpkTgP5yAlHKslnWijfZT7H9XP9D8cHcKLrlvV3LBpRXC2ywMf7Rh3rxISZUl+sLeQ6DFnkhCO5plSXwuOtnzHMtiVncnOYjuvn+h+/NIREMmX9/VTrg9b2W9Y04b1TPdb7J7pu4XB8oOyEPAxZvIXKbCUhJ1zx+8PLRxSDmgD8jmE2XUjy2KIRZose0Y/r9frrIrZr0wplH0+1PoijnWkLNZFMBXqOpTBxA5VdPItcK0Sg5Pp4G2QySinHkMtjS6Y4ogyux/Xq8sn22L5r04qCnGOpJRG5lRcuZ8giJwIl18fbIKxi2WVRWx21jcHJcpxNS06/Pm4Lj7L1/t6pHjzV+qBrb1A3671Q56h/b292XLCOFzSm769UngaKDYUfEoGTy82Vb6hjtsiMUgqlPBwfwKGT6TA0N2HWw9UAFH3sOvJ1FRRijKbvD0DJfKezhVP4IblWiMDJ5fE23xV93TIcm5xSxlBqEQ2fXRnG0c4hV3eE7PIRlMLYZcT31tZcZ71WiDGavr9S+06LCQk5UTLk49/M5ucutB/cT/hiNgES+wKAfS9swLa2JsSikYKNPV92tDfj+zsfLuj1NX1/lRQemw1yrRBlQzaXjp9Udj/HAvw94mfLwDS9l68veDZ8yUFGHZn2Qz5yStEnCIu9HZ3Y//FlJFM8Z3/8qwfPIJFMIRaNKKF9gLfU670dnTgS78f29sVKinwQadym8YbFlxymsRYD8pETZY8X94Zb9T+vHDrZbSXXJJIpXB6667sC4YFPrqJzYBwHPrmqjLe+phrRCMu6ryBdOfnuP0jI750bnoWcMXaAMTbIGDsvvdbIGDvMGLuU+f+CwgyTINzxGs98/NIQktJDaDTCPPtWhbjdHL+vvH799j3s3rzS80Ktk1gJgRfx5bs3r3R0EfmJ3c6l1G2xYsPJ750bfizyvwDwLe21fwPgQ875GgAfZv4myoRiWWW54NWSk4UiyoCdjyzG8UtDWc9RFrfO/nEw6b0Uhy1KxvR5cS2dxEpJFsrsM59zFfiNCCqmVRzWuvDFxnNCEOf8GGPsa9rLzwF4JvPvvwTwEYA/CmJgRHEp9UpyOl4TkeTEmGy1TuSFNFncEskU1i2bh/iNMcvPnq2glX4t5THIjR28nEMuSVdeUv1zTagKmkLVIyln8s3sbOac38j8ux+A49VnjL0C4BUAWL58eZ6HJQpN2ErYumUuigScofEEmupi2LUpvYD4xvvnHc9RFt+DJ69h5yOLFXF77dm1ALxFvpiupVjA1DM3vZSYLUSWplNClZhozvaMuE56RHEJLEWfc84ZY44hMJzznwD4CZCOWgnquERhCLqSXBDWXLZ9mCw5OcJEcKLrFt568THXc9RronScv4E9z7TaRDaXglYmV0oimcLRziF8dmXYk0gGbbWaEqq2rGnyVeArXyotlDBI8o1aGWCMLQGAzP8H8x8SUQoE6avMtnjmNdoklwU4vbwrkBbNQye7Xc+xvqZa8YMLn7Vws5iO73QeTsfZsmYm0UcwGz5p0zhNfnunAl+xaAQ9w/cCXTspteJbYSNfIf8VgH+R+fe/APB+nvsjSoigKsm5LZ75iTbJZQFuy5omVEWY7fUTXbewt6PT0RVz4JOrkB8ba6ujqK+pdhxrtvMwXUtRXlZGRNEUaqHZaZymyUYX9z3PtGJbW/ppIlt5Ab9Q2GF++Ak/fBfApwDaGGPXGWP/EsB/BLCDMXYJwPbM3wSh4BZSlku0iV9XT4TZhTyRTGH/R11459NuvHrwDF5++5QlSvKYAKCtuQ77XtiAsckpx7HmKkRyeVkRRXPoZHpMhbBO3capTza6uL++sw0tjXOsJ5wgBZfCDvPDT9TKCw5vPRvQWIgyxW1xLpdoEz8+VN21Iny9ss9X+KdPdN3CU60Pon1pg7Kw+f2dD1vHM431cHwAPcP3UBVhmE5xpc2an2sjR9EITIuw+fiR/a596L74XNdOvKxvVGpTiCCgFH2i6PgVp1zbuOnRGLpoCtzqhOvHNpVxjUUjeOvFx3yLUbaytUGlrzuVB/CK35onxUi7L9eFU6cUfWosQQSO35vITwSG3/h2N0tvfct8HDrZjRNdtxSrXS6Da9ofAOz78CJ+8KvzaJwbs00GTq3Usl0X2doVNVzkeuVBNd8QE9i14atY3zLf9z707yvbdzLboaxhy4EIAhJyIlAKfRPlIgpOE4V4XcSZC0GPRhjqa6qt85GrHB462Y2PLw4h45VB78ik5VIRyJ8X6Ndl28OLcHlwTLGKd7Q3Y/fmlY7Wsr5P/e9sHI4P4M2OC4F1YhITUrbvZLabIoctByIISMiJQCn0TeRFFHJ5IgCAK0Pj6B6eQDLFceCTqwBgWa/vneoBAFsoIwDMq63C6MS05XMXnxfWrklAPziXzqPrHOgCALy+sy2rtayn7B+J99u2cSsBa3IBiTDCfJ6Isn0ns+3/nu2JoxQgISdyxiQahb6JsomC0xOBW3s1U9LQxFQSR+L9SsKOE8vm12L47h3lNTmiQxdQnSPxfry+s82XZQsAnQPj2PPzz7Fn62prInB6GtIjcZbNr8HQWMJXEpK+HzlLNZtQz2bafSUunJKQEznhJBqzcRO5iYJTeJ0s1CKzU2zfM3zPJtRRBmxvX4xrw1ctnzUAqwb5koYHMHZ/Gsvm1+LptYvQNXjXZu3qSTVAOpRx9aJ6yyIH0scB3CdBMWkK10vnwDiAtPW//6MurG+Zbzt3uRGyvu+1zfXoHRlSrpPfLNUom3HvyG6qN94/X3QBrbR6LSTkhA0vrgk30ZiNm8gUPeJU8EkPQRSZnZ9dGcbEVBJVEQYGWAlAEQbseaYVr+9sswRS7yp0tmcE+z/qwvDdKXQN3sW2hxfhy2u30X9nUimTqwuoCGU0RY44TYKmyJuuocszrhw+My7dYn/t3S+sSVYOc4z3jSIWjSCRTPl6chJ+fNGYQ3chVdoiY6lAQk4oeL0Zs4mG32NmmzhkoY73jVoLk391+jp2b15p+ZZj0QjaFtdhYd0DigvlvVM9lpgL61qMXSxURgC0L2vA955doyTF6NmYezs68WdHuyzhn5hKouP8DUXAReSK7nYAYFmsptA/0yRoqoOyZ+vqdIMMPpNAI8T6zY4LlsUuW9tiv+L7rYowLJtfg+c3POTrOxubnDLWXanERcZSgYScUPB6M2YTDa94mThMC3UCky/7XO8d1FZHsWvTCmusb734mOIjB2BZ5IIUgMeWz7dZ+HI8uegwpGdfJLUXZCtXdjt4OVcv6w472puVpwV54gHUHqIi5V+4keQJrHdkUrGqveDkAqrERcZSgYScUFB8oIYwOhkn0QDgKIQ6XiYO3c8sU1sdxepF9dZkItD3JYupEL99L2xQwg5l0dMnDiG8xy+pHYYYgH+ybgmOxAeQSKZQFWHYsmahbUEVSMeey+d66GS3co1ujifQ8VU/kiluHU+cvylByS2sUn8KEOcTi0Ysl4rTtcqGkwuoEhcZSwUS8jInl1C83ZtXWn0ts1lrppvXTQgBtYa3FytOd+MAaT/2I0vn4bVn1xrrfehp8nqsuBjP2y8/brtGcp1ygVgH2N4+U5c8mvGlr2+ZjyOZeigRxtC+tMEak7huezs6ca5XjWw5fummEn+uH0/24/vNiJRFXj6fRDKFbW1NuDl+H1/13UGK51bbJFtsPjG7kJCXMX4Xn+THb2F1OtX60C1tp0w+gS5M8ni8hK6JbW6OJyx/dNfgXQBmoZ9OpnC2Z8RxYtF9x25hfoLOgXFcG75qS+Z54/3zSjNmsRAon+Mvv7huOy8nEQdgNWAOwuesT5btSxtw4JOrSHG49gYlwgMJeRnjZ/FJFjv58Vt3lzhZ2tmEUFS2M43HixUntnnj/fO2SUYsKMr++hSA/R9ftoXlCapcmi6LiUN0Fbo9fh+9o5PWMYX7QyTt6O4neSHw0MluHDrZjRuZz8tEMuM0vd6+pB7tSxsUizyb1ez09KVPlvL1SHJzYhERLkjIyxg/i0/yzS0ev1sa5yii4GRp6xOEHuomLPezPSM42jnjBvGbYu52TuL4e3522hL6ZIrj0Mlu7Nq0wm6xp7hlsTshRFSe2KIMtoiNnuF7xs/HohFbHRf5ve8+vcoq4CX+LyJyzvXeQdfgXZtv3EvmptPTV8/wPRw62a1UdwTyizgiSgMS8jJG+LtFvLLbTaoLpIjskH29JktbpHnv7ei0uVpMi5YyuViC2Vwx7Uvn4XzfHYiinie6bmHXpnQ97X/93he4m5gZ+1/+/VXH6n/6xLZsfg3WNtdbbglxneprqnFxYMz2+WiEYUnDA+genlBejwB4ZFnat28677SbZiZRRy7e5TVz0+QO+/2ff265ck503cJ3n16lJBZRuGC4ybdDEFHCiNodnQPjOPDJVdcGBUIgRRMBAEonGdFNZ/fmlXjpyRV4dZvaLeato+kmDXt+dhp7Ozqt48tdbuTmAcCMJZitxZveKWdHu73bjhC5c713AEM89472Zqxqmqvse/x+0hqrjj7W3pFJfHZlGOtb5lvXScSv947Y3SbJFFdEPMLS/6Uw49vPdlw/TTi2rJlpG6cv9B462a344xPJFMYmp/D9nQ/n1MyhUN2LiNwhi7yM8Zug4RTpMDGVVJJPhCUoW4+CJE/7pgEYu657iT2XF1S9dm6Xz1VeQoxFI7g5nsDOP/kYqxfV2yJHxFOBycJ/YlUjLvaPKf7xQye78fbLjxujWxrnVmP4rlrYSrCkocYSfDn0MJs/22ntQRfesz0jmE6Z68EMjSeUv1lmX7mEC1L2ZmlCFnkZI1t3fhvmyp+NRphtgVHfRiaZ4kqSjvyZHe3Nrpag3FNy/8eXHS1Qmb0dnfib3/TB3tANWNxQgw/O3UDnwDg+OHcDqzWrfPWielsPSzGGo51DGBpPKD0/T3TdMj5hRCMMy+bXGsdXFWFYMPcBxWI+0XXLV39P8bqpibNIUhJGt3gKETTVxZT9PLpsnhL77acvq9tTgVfIog8eEvKQ43ZTiBs/l4a5smjs2braKLzyNt9et8QKmautjlrx1vpn9M+5NSGQO7c7Pfrv7ejEW0fTNU+EJS6ENxphGL+vWshVEYZXt7WirbkOr25rxURi2iZMun98Xu3Mg6sskmINQiyAdvaPw9DnGakUx/neUQDAtrYmPNX6oOe+l/r3axJePUkpKkXkiM+JaxKLRvDas2sdj5cNJ9ePV3H22myb8Ae5VmaZIFtQeXnM3dHenCka5b/SnbyQ5tT6THbH6IWgnFwW+udkdPfB7s0rEe8bdRzjkXi/7bWvL6lHvO8OkimOOxPTyntibKLs60+PXbHek33LcqceeR+6/3lscsoSURHtc2VoHNeGJ8ABW2/QlsY5qK+pVqJ3bo4njBUDc617s/ORxbb4+Vg0gm1tMxmnuf4OsyWAZSs9IJcIcHOrUWaoP0jIZ5Gg/YtefeC51sDw02tRborQNZT2kb++s811ey8+YmDG1y7qZp/tGbEmDD09P8KAhXUPWOI6neJYt2weEtMpW9cdvSriU60PWmMRY+gZvqeIrryN6drqQin7+YV7S/dZ/+25G0jBHpOf7fuVr6GVjcuBjq/6rSgi+cmipXGOZ+F1wy0BzGmc2XIU9O3I/+4PEvJZJOjqcF4F2iSOXmpG+xmv7hIRNbJN25tuWCAdXXFz/L5SufDlt08pY/jRh5csN0XnQBca56qx6FvXNmHXphVW/LZwJTgtpop4agbgy+sj2NvRafXyHBpPgIErMeTtSxtsk5BT3RHxb7k36NHOIZs/U0wlogyAoGf4nnVsvZuPfg2fWNWoxM/v/6gLe55pVcoJiLj92f4deslRKMS4KgkS8lkk6OpwfqIOhLD4sXq8jvdwfAA9w/fS4XVCTDI1sk371m9YUwNkEet8ouuW9VosGsHt8fvKvuQoETn+3Qn9KWPbw4vwP87dAM/s662jXUptcmAmdDDJYbli5Hot2dDdW6nMPlM87buOMGade+fAOF49eAYpzjGd4qiKMKxbNg+d/eNKNx/9Gt4cv6+4cZI87faRa4eLaKLZ/h06PbXoUPXE3CEhn0VyCffysk8/+/Fj9XgZryyMVREGcI4U3OOS9RsWsLdSSyRTOBLvt7k+2pc24K2jXcb9bnt4kRQWOVP7RD7HQye7lfO/PDhmK0mr/53S4tIF+iT03qkeq/PQvg8vYvhuAs9veAiv72zDTc2d8q1Hl2Aikfa9ty9tUJJz5GNMpziGxxO2xdEta5pw6OQ1Kz68s38cOx9ZbNWhkZtqyL1E93/Uhf2/u3FWf4def/detyM/uh0S8lnGr/A6keuP2a/Vk2288sQwneKOj836uOXFUwA2izwWjWD1onp0DY5bwiQsuc+u3MLn3bdt+/6y5zbeeP+8sUvQzLFvWttHWDr88NLgOFzqVynILd9qq6O4OX5fmTT2fXgRv70xZgnsW0e78Ddf9uKaluE5kZi2SgB8dmUYuzevtNrK6TVYFtQ9gOF7U8r5/OKLXluSz8K6GPb/7kbld3G2Z0Q5rnhS8hNyGARef/fZtiM/uhkS8hCSz4856KcCL4/Nsk/6p8euWD7ft158zNpWNH4QPnKRCp/k6QQWYW0fjg/g7DW7iAPAjdFJvPNpN2LRCNY91IDLg2P4R60zkRU/+NV5RfxSHDgSH0CEMaQyOf3fXLEA566PGuujtDXXYXv7Yhy7OGhZ23pEzfDdhK2qoS1NPxOiKD8ZjE1OKfVp5Ov0vWfXAFDbzMk9PwX1NdU2IRybVMMvI4Diaw8b5Ec3Q0IeQvQfs9wv0wterCOvFn+2iUGedGRLM5FM4UcfXlI+J0d7xPtGlUzNjvM3rPf0bjwCOSFGWOwfnLuBf3XwDI5eGLQV/BLbysyrqbImFbleuHhC+LGUePPTY1csP34imUIEwDdaFmDgTr9ridr2pQ1oX9qAYxeHbK3a5AVS0wIqAPzgV+eN+zXVjq+vqbZ852lfP1N87V5/M6XiziA/uhkS8hCixw3nW73O1MjYj8XvNjHIk45u437VO4rzvaN459NufHPFAsyrqVKiTUyLp061wt34+y7nDkOxaMRaWEyP9yZ2bVphLZoOjSfQVBdD+9IGJXsSSE8C8b5RrF1cj3jfKFIcOHphEL+3dTWOXRxE78gERu5NKZ+pijBsXdtkPW1EI8xYD1wOQZT/BoDGuTFjfRfhsxfbipBQkVjVvnSeVaLAjzVbSu6MQqwzlQMk5CFE/Jjz7ZcJmG/SfC1+GVl4ZWEG1EVF3eedSKbw6LIG/LZv1Nhg+PildJMJk4tBjzpZvSidICSLOQOwNNN4+NjFQUvgplMcf/TXX2J8Mmn5wb/3rL3FG5AWZd23PzGVxMcXh/A3//sWy50ji+7Xl9Qr8d3JFLe5PwD79yKvKTy9dpGtZoxAlBAQkTLWcXg6vt60dpCNfN0ZQVvzQa0zlRMk5CFF/JBN/TL9YLpJTRb/qwfPWFEjXupjy+OUmzQIy1UXdR3ZNyyaJsv7FNEpOqL92tWbd62wwnjfHezevBK//OK6JaocsBoPty5S66/IIY2maxIB0L6sAU11MSVZSHC+d9TRndPZP46WRvV4prrs+vcidx16YlWj+aJBjdKR3SoiBn7XphW+RTUfd0YpWfPlDAl5iAniMdN0k5os/kQyhaOdQ5ZwCSvRa3VC3XKtiqQXJE3RJzpydId8DHnssWgET7U+aC22vvH+ecsqF4uJJpfExFQSHMxxYtGviZhU2pc2IN43amtkLHBy5ySSKVweVOuXx/tG8fLbp6z9ikbMMnIzCzEucd5ti+vQ2T9uayAt3CpA2iI/8MlV7Hthg1XfXMdrtm2hwl2J3AlEyBlj/wBgDEASwDTnfGMQ+yWyk+9j5tmeETTOqcaCujp879k1toU1vbWbYGIqaaxwaBrLoZPdxjjxR5bOw8i9BC4P2etzy8WpTMeQwxhF5Ijc9Fi2RoW4mTr5xKIRXOwfSz8lAIhEGKZT6WzOtYvr0VQXw9meESuaREwqYkKLRSPG8rX/qLUJR+IDSCRTStKPKCgmQg1j0YiyqCr2G9Wqb4mJJhaNWD58fV1D/lvOiJW/M6fvKJvlnOvvjBYnZ4cgLfJtnPOb2TcjZhO9kJX+nkiu6R2dNLY+e2JVI26O37csPoEuSH5v0ghL3+Qnr9wyvi/vTwhBNMJQX1OtRsJkIjGmU9wSwfdO9QCYqZ4oLyaKJ4N0z86FAGbEMwVg65qFVmGrA59cxfneGdGOZrI7ZRLJFGqrYgBUIV+5cMZ9EmHMausmhFZEpei1XATJFLeifMSCrAiRBOzCKv+9t6MTHxv26fYdFcpypsXJ2YFcK2XMvzp4xloM7BxIC7Ys5nrlwCPxfuv9w/EBvHrwjBVB8t2nVyHeN6rUQgFg+Wud0q7Fe7pYPbRgjpWwIsYGpGtnP7qsQdmfVRAqxXHgk6t4YlXjTCQMhyJwgBpSKFLVgbSovPXiYzZLVm5wLLtmdItWRJkkJR9MbXUUzz/2EP7saJeywPqXf39VSRQ6Eu/H93c+rDzxiOM79fWMRBi2apON7AM3uUIOxwew/+PLSoTQumXzsGH5AlchlS1nuS5LEBRicbJUwiFLhaCEnAP4O8YYB/BfOec/0TdgjL0C4BUAWL58eUCHJZw4HB+wRXTIQg2kS7rKIrq9fbF1g3xx7bYiRMcuDqJr8K4leHrvSuHWcCpZu6KxVkmMacjU+F7fMl/xTw+NJ6wQQIFcKlaIqy6oMrorQ6+F7sX/axKyaIRhz9bVSsNk8Rk58gVIt5GT6RwYx+///HNsWbNQmaR2tDfju0+vwo+PdiEFNeJmOsWtLFk5LFP4wMVTyXuneqz1gUMnu5XrEmVw7A+qi6Fcl8UUk14q0AKqnaCEfDPnvJcxtgjAYcbYBc75MXmDjLj/BAA2btzoMSGayBVTs4Lt7YsBqDfwq9taLdfL+pb5istCZvhuQnn01v3jooSq6cY6HB/A2H21LjjP9PM5fmnItsg4neJKPLS+8CcSasQxdYRYuiUpyZUgTduZQgJ3PrLYqmOuf+a1Z9filXdO2+q06OdlSsaJ941aFrRcw1yehMRTh/i/XlHwaOcQTnTdUp5OIpkIHq8+8bHJKWVRVfyGSs3ypQVUO4F0COKc92b+PwjgFwAeD2K/RO7U11Rb3XUA4NvrllgiJHdoWd8yHx1/uBWv72xTk3f4zI8jFo3g+Q0PKZ1h5A5ATq3ggBnB0BcDRfsxp0f4m5kqh4fjA+j4SnUBjU1O4fWdbdj/uxvR1lynvBdhM24eEZ0hd66Rz//Vg2fw6sEz6abRP/9cacS8ZU2Tcv0AYGFdzLXDjb5AKcajI18j4VoRxKIR7Nm6WumeJDdPFpOc6bolkiklo3Tr2ibbuojAKexU/o7ra6pLspuPU5eiSiZvi5wxNhdAhHM+lvn3Pwbww7xHRlj49QdaoWd8xh0gbmg3a8atO8/6lvnYvXmlsQOQ3DzBrfO7QEReAGbLF0gnr4jP664CsX+Tjx1Q4871pBr5SULxpac4fpxZ+BUukz3PtFquBnFeevVEcf2OXxpSRFT4peW6KQL5GqXL26oVHp3EV8Z03SIsHdYpXEpuJX31iUDUaZHdTKVq+dICqp0gXCvNAH7BGBP7O8Q5/9sA9ksgN3+g3uRBLuzkFg6m3yDATHceYTUmkilcG57xn4pFN9Oip6hTLjdHkGO9xft6NEhVhKF9aYOtkmEkk2Yu9i2iPmRSHJb7Ql4Uld0/8nGSnEN4I1KAVUtFXOs9W1fjl19cR+NcEYYoVU/ETJs2veLi02sXYWxyCutb5lsLrLpf3fR96OJ7OD5g6yjUvrQB61vm471TPdYkwAA8snSeddxsAqdPBPKCsPy5Ug0dpOxOlbyFnHN+BcA3AhhLSVEqq+K5WEVb1jQpN7nc+V0vIWtamBSvyZEbehq6HDkhZ5cKIdJDBNctm6csusnv6yyYo1b/ExEzJ7pu4VzvHbx68Iw1plg0gqpM7LfMxFSm2QIzR5sA6bDABXOqFaFMSS4iudZ478gkzvfeUXzgKcBaUBZPMGIhVE6U2r15peN35WZdOl2jsckpKwJHVIzs7B/Hud476Bq862my9xLfbRpbqdwXhAqFHxoopVXxXBIqdrQ346nWB5WQtUMnu5UwO/mcnG5OPXNS7Et3DTi5GmR/e7xPrQ1icrkIZGEVhalaGudYHXbkSUW0Drs5fh+9I5MYuZewSgB81XcHqYyIr2+ZjzPdtxUhTiRTuJdQF2FFBI2p4YXbQubEVBLxvlG8/fLjtvZ0+pOA3FlIXHdTtqXpGgnftWjV9/bLj+ON98/7Lobl1T0hT+yldF8QKiTkBkrJN5irP3DXphWKcAPOGZJON6d87PqaastFI7tPnCaa+ppqJZQuyYE3Oy7gbM+IZbUKV4QX3CYVORRSIBvfyRTHF5qIC+4m1GYWcuIOoJYW0Itx6Ry/dBN7OzqVxcuIIYloYirde/Ty4LirKJpKEMjnKj7nd7KXJ255AslmbZfSfUGokJAbKLW04lz8gSZ/tyzs9TXVePntU7jYP+Z6c+qp+nLMuPD56q4asdiqi17nwLi1MCm7Is5cG7EaKuvIUSi7N6/EL89cx4K6B7B1bZM1sci1y52wp9uotDXXKQk7AuG+ANK+aeHyEZmhl4fu4lrGTz+d4viLE1cVK/6RpfOs+HuZr3pHlVowcrilwDSJy+4u8X398LlHXd0zXkoUe7G2g74vyE0THCTkBsK4Km66KfQJwNSBRsbp5rRV4tNcBcJVA6T96j3D97IKqyhk9cPnHnX1l//+M60AgJffPmXVJOkdncSFG3espJ9YNOJYvMoNYWFXRRiWzq81biOunwgXfKr1QQAzk8vLb5+yhBwA7iZmzqG2OorXnl1rff6jziFrWw7VwpfLz+rHd3J3yd+XabL3UqLY5ApzsraDvC/ITRMsJOQOlOqquFNatpebQpyT3JxY4GSRAlr6tmHRUFiUwuL3Iqy6CAmB+KrvjlURMRaN4OrNu1YIoEx6cTP9mvCTixop8b5RS/Td3CFyBqWcqANAeZLRJxl5UXfXphXGFPtlDTX4gdQX0yT60QgwnflYttR7gZOYmj6TrUSx/B3obhyndnB+7gu38yA3TbAwzt28foVh48aN/PTp07N+3LCjR4gIwX7j/fN459OZ2OmXnlxhWzzTsxlFHRUAtv6ZpuPK7gXdH11bHcUTqxqVeipzY1HMiVXh5vh9cMwkxojFRzm2Xeblt08Zi0jpVGV2KCoV6uMXY/71Pwzb0uXd2NampsPLi8Yy8jU2NZB4dFkDHls+Xwk5BIDf//nnxjZw8lON6Tt2w+l34fa6kxtGjtTxcny3fbmdR7b3CTOMsc9N1WXJIg8RTlZMtoJHJotd9v26FbzSb7j2pQ1WzLjcZAKY8cEDaReD7GbQFx/1psXiWPpCofy5CEv7nEVz5p8euwK35UenYlROxKIRfCk1Xk4kU8YSu/o1FolJIga9KsJwsX9M8fv/7NNu/JN1S7BlzULbxCA/DZl84H7yBuTPOFnvTla1cLGI6KBsx3d7EsxmcYfRfVnKkJCHCDf/qF7wCJjJUDTdVD/MPPYLK+zQyW4lUccpu0/OdNQngN2bV1plcbPxcecQ9nZ0KtUW3+y4oAhv+9IGK7IjgrSIi1h02T0kuyUEesYkAKxorEXvyKRlEYuJoirC8PUl9ejsH8fwXa2hg7YPhnQUyv6PL+PqzbtYWBez4sZTPC3yX19Sb2vFxpGOOf/2uiWK2ykWjWB7+2LLBy9KK8jt7bKRLcnLj0j6WdD0kyXsFKdOAh4MJOQhws2K0QseyUWsdm9eaezVKJeqBdIW7HefXmUthL53qgfffXqV9dkog62okj4Gr6SQFkMAik9bkO6VOdPqTU4GEmF48rj0p5Ata5rws0+7FXu9TxJxIJ0QtHXtTIEtXXxj0Qi+sXwB+s/fsBZ3rXDKFLeSgeR1A703ps6X127jqdYHrXLAsqvKqqOecT9te3iRJfBuVvHxS0PY9vAiXB4cw/b2xXmJox9L2U+WMAl2YSEhDxlOVozTgqSIDjHdVLrVmkim8Msz1xVLN943qkS7ONVU0ccgcFtsTKa4sYKhvFB4OD6AvpEJZUxiYXLbw4vwtxmR/emxK0rZ1R3tzZj7QFTxj08ZFkxbGudYn5GzYSMM2N7ejKMXBo0VFvXzkK1ouYOPvHgLAANj99E7OmlFtOhVDOV9dmTOzWkB2xTtI5dPyBWvlnI2sSaLe/YgIS8T9OQdXXDlm0pYcfU11bbH/AV1D6B3dNK2b/mG/OUX11EVYZaP3SmBSPafi23vTE5bGZZONcXXLq63LdbpTEwl8dmVW5YPXWSvyuNcuXCuY7d5cb6ye0pe2Exx4PLgmPHYpv3oHYDk6yK6NFVXRS2/uSmKRHQCmk5xJZHIyVdtyvwU24r33Ur1BkEhxZrizL1DQl5GyDeVqEyYLZpge3szvuy5jca5MSvmWe4MtGvTCkX45fjz7uEJnOi6pUSMuC2kiWNzpH3JOx9ZbOw03760AYBdqBrnVOP2vSnLwr+t+bN1Xnt2LX7vZ6eNTZUBYN1D6cQmkW3avrRBSZqSW9kJ5OgbGTcr+PWdbUoJYf2JRl48Fgu4EcZQFTE3xxCYnoDk8rPpsNBrSKU4UnC27EsRijP3Bwl5ESmkxeEkqPoCVcdX/UimuFUvfEd7sxLRcrZnxLLuRQ9JGdNCo45cqVAcO+1LjmHfCxvwZscFdA6MW9uLiBa5+FdVhGH8flJx03BA6WupVw7c0d6MrWubHMMZP+++rbg95GxT8Z2ICfHmeMJydcSiEdTVRK1rlmsMOKCGGop9AWm3z7a2hVaHIKfJ0fQEJH/H8ppAmOK1Kc7cHyTkRSJIi8PPhKCHKjotXgrL9Nilm9Y2pkC+dKq6ObJBRMSIhcyqCLNcObLLB1Dj2o9fumlrYiBcDjK11VFse3gRvrx2Gwsy9cv1eHnRoMILE1NJq0CXQE6iSkpunJF7M1eDIR2F80//9LjVpNrpO5UnWD3UUJyTWOcQ5WrdFjydJmzdUgfUWu6lTqmVySh1KCGowDiJrJckHi/7kicEvd53tv3ovnSRkKJbyKbu8QKnZCIn/7ZTI+B/+qfHFX+2yNSUr5EYR1WEobn+AXxj+QIciQ9YE4Dcq9NpkbUqwjId6c3vyb0+972wAWd7RnAk3o/Vi+otN5Db9ZCR48OzfXfy8cQCsF4czM9kryf3uCVhFQMvxgf5yO1QQlARcLO6TR1actmXHvXglmou3xQmXzpgTkcXnYKEKMgimUim8GbHBQCqxehUpnZh3QPGCUt0BJLRrTJ5HL2jk+g7d0MRazlt36SzjXOr8Z/+Wbp0/qGT3fioc0jZrrl+ZqF3YiqJfR9etCaXzoFxLG+cg9VNc43ZrSY6B8bx2rtfYPfmlbaKhU4JO8cvDSmWv0CUQfC6gCn2X4pi6LekBJEdEvIC4ubnc+rQ4ndfpglAr30i4pMTyRQOftaNPc+0WpaZ06M+YK+/srej0+r2LiMES74h9eYWQNridWo/JtcsEf5uk9ClE4HSrgZdrKsyUTBOxvLtu2oXnL0dnUqC0/OPPaQ8oejJQdeG76H39j20L23Avhc2KG4jmQeqGO5Pz7is9EbV8sSni5VTud5YNJK5PkPK95nNLVeKYkj+7+AJpPkyYWbLGucmsW7vAWmrRW4abNr+cHwAR+JqY2Jgpg6JbKkLQRVZiboPWm7LJo6hF9GSu70D6XoqAjnsDZgJ51Ovx0KjC+blt0/h0MlurHuoAfNrq9DSWIs3Oy5gb0cndrQ3W1mo4jroTY6XNdRg3bJ52LJmIZY3mqsYAmnhl/t5vr6zDfu/802r0fHrO9uw74UN2NbWhCdWNWJxg31f4voBwNsvP44ff+ebWNZQo2wjRBywN6oGZiY+UzNjMXm99OQKvPXiY3jrxcfw0pMr8FTrg0osvfi3ft3DQLbfPuEfssgLiFvChHhPjg4xuThki8sp4kEnmeJKKF0sGsF0MmWJcDLFFStI97Nva2uyLGfRicZkMS2siyE1lnBckNKbW5j6UcqLnIKRiXTXHlG7XITuiXPf+chiK6sSAL6xfAGOXhjEud47jq3f/CDXjDGRTHHs+/CiYu2avgv5iWZ9y3xl7cHNEtWtaOEikb9PwN6tKSxkSyQi/ENCXmCyPdqKm1OEyP3V6eu2psFyESQnN4gMB2zZnGd7Rmwd4QW6n11EbeiTSTpbccaVcG14Assba7Gqqc64wGqafESj4rHJKXxx7XbWolZH4v1Y3zLfGsvBz7qtBsyCv+9Sx//osgb0jdyzwgMF4klFlGfVfbWicYXpuq5onINrw/cst8253jtWrRhxnrJPHQBWL6q3RZyY4si9YLqWYRbCUnT5hBkS8iLilJkHzISh6Te8HHEitolFI5hOpaxIDJGxKN8scky0fvPri4r1NdV4s+OCza/7/Z0PK9X7ONJJQTdG7zv6vgW/+KLXisP2w+pF9cpYkjzdi1POSBUWPJAOBYz3jdqiUubGIphKQlkM1n21f3a0y9G/vqppLqZTKaVU7cHP/sFKBBILlbKQm1LsTRObVzE2WeoEAZCQzxqm6AGnzDy5VoduycqLcXLyCgD86MNLuD1+H88/9pDxJj/bM4KTV26hvqba6ObRQxJl5AiMYxeHFEEWCTGAaiW6pdg78UCUYV5tDLEos1wm+udTPN2pp29kQgmTBNKTiymidmVTvS09Xl8o1j8monNi0QjalzZYDZ4FIxPTyiKvW70b/amKMheJICEhLwBe+yQ6ZebJVrT82ahWf0O0ShPHFCVfD3wyUzhJjOXmeMLyK8u+Z4Gc+CILZyzKkEiqBbj2PNNq1d5ObxNR0sJNbcW8MpXiGBq/j9rqKHqG1V6XsrDu2rQCZ3tGrHMxEWHAkoYaPL/hIcU9I55y3BYJqyIMv7d1tdUoWkxuVRGGB6oiVq11XaRNE6Kp406QkRulGGJIzC4k5Fnwe5P46ZMIZPcVyp8V5U1Nfu5DJ7ttxwCcF0SPxPuNySH6U0IiqUZgyKnrcmMK0/FNTxxAWpAfXTYPv70xZluUTEkT1Vd9M24KuSNQinMl+sSJrWub8PbLj1t/626NnuF7ysJohKWt+XRHI2ZNhvLkNp3ieLipTmkaoTeZkGP0RVLO0c4hnOi6ZSVsOeURBPF7IzGvPEjIXZBvkvdO9XjKmjSJdj7pxnpK/c5HFmNhXcyWIXj80k3lc19cu+3aBHl7+2Lj+R6/NITdm1fiSLxfcVvoMeWyYOnHF2n7soX6cecguocnAKSFsvf2hGtkiZ49Oa+2ylq8FD02q7QwRBk5SkZuVSdek6N01i2rtxKShP9frp8ix8THohE01cWUY8X7RhVftyzGLY1zrLh3OWFLFMoSjE1O5STKFJNNACTkrrhlTZriofVFSNmCzTXcakd7uvvPjzNp20fiA7aU+OOXhmyiKELxxKJgLBrBkoYaTCdTeP6xh2zWuJ4uvnvzSqvyXzTCXBsWHDrZrRxfLk0rBP/lt09ZQg4Aw/emHBs0tzXXYXv7YsU1cUda0BSYJgK9BICpecZTrQ8q3+vCugesxs1yuKTThKuHdopIHtGIQ1SIPHTyGr6+pN52nk4L2rmIMtUkIQAScldMrgHTDWYSQZO/O1dLSY7CMNXd3rKmCYdOXrMJm+guD6QFrHv4Hmqro1jfMh/ATJ3s7e2LMTY5pYjI2ORUun3cR11W+zivDQs4YEvbb1/aYKtCuLihBr0jE4rwx6IRLJ1fi/Ut863Jr2f4nrGCoexuEZ8VreAEpuYZFwfGlAlOCLFTXXF5H4lkSgnt/OLabStSJZFM4d1T3UoFQzGhPrqsARf7x6zYb31BWxzLryhTTDYBkJC7IiftyJ3F9RtMt6TkRch8ORwfwMX+sazbRVh6KZAxIMrSfl8hGGkhUhvqnu0Zsfprdg504dvrlhgtRKfmBvITCABbiVs5bR9Aps62SjKVUkR8WUMNhsYTNn8yYE7S2bJmoeWfB8xNpE2lAnpHJq3Ep6HxhOXvTiRT+PjiEB5bPt+2D11g5SeNbCSSKTy2fD6+9+wam+A6RQ/5EeVSicmmRdfiUfFCnu3HJ4eLOW1XqMdbU/ieqe62bDFyDmxZa69jrY9PWMyCy4NjNhE52zNibATsNaxQiH/P8D2jC+UbLQswfHfQGtfaxfXo7bT7k/e9sME4oYrroJ+r/l2J+uoXB8as8EE58Ukm3juK872jxugi0wKrXiPmhcdXKM03ANjE341SEWW/0KJrcaloIffz43O7wfJ9vHWaJPTwPX3BUVBfU61Es5gsU7G4Jt7TQ/eED1wWwwOfXLUiZXZvXmkMnTOhi79ThIloLCFHkuiWt5gMfij18HQrZWB6zRS7LU9MQogjTI2a0V1oYmzyOomYKOTvTyRemUJK8yFXi3c2LGVadC0uFS3kQf74/FpSpprg+mSiW/omEbcEN9MAWBZc8b4sXsKKFYudwkeuL34qYY8prlRndAorBDIt3B5dYuvoLsRSYMo+BZDVlSVvrzdmECGXbqGepglXCLFbc2mn8E6xL9mVVgirOleLd7YsZVp0LS4VLeTF+vG5JfmYhMdk0Tq1T9PL4bpNVqKXpAm3ayMLotwCLRphSh9OuaO7cG+Yyr7KeHFlZRuj23dqD5tUhdhUxuBwfAAnum5Z+3BKgHIaZxAWca5Gx2xZyrToWlwCEXLG2LcA/AhAFMD/yzn/j0Hst9AU68fnNclHoD/SA2octN4+TUZ3u4jyt9nOOdu1cRJEJ+HY0d6shClm6/Xp1Z9sGqOX71QOS3zvVI+xy5FAj3x5qvVBW5SP3PTB6YkoH4s4V6NjNo2VsPr3y4G8hZwxFgXwFoAdAK4D+DVj7Fec83i++54NivHj028uU7iiwCSM4t/ATIihqUmvye0CmH3IJrxeG307k3CYrFpdVLy2/9KjVNwmGScOnexWQgp/8KvzSqNp+drI35coIasXLRNNH/RrGpRFnE9EC1nK5U8QFvnjALo451cAgDH2HoDnAIRCyIuBn5vLi/vAKdtUt/zHJqcK/qjtdG4mq9av5WpK7nnrxces/ZsmMq8C1jsyafXKBMw1VH704SXE+0ataBoxActx7vo1DdIiztXoIEu5/AlCyJcB6JH+vg5gk74RY+wVAK8AwPLlywM4bLjxY+3qwng4PmCLQjGhW5I9w+k2ZU4lct3wI4qmc9MFzRRCmW2CMSX3yC3t5AnAaWIQ51Ebq7LFvmdzc/1WSsyS8wXkpg9u6wlyJUuyjokgmbXFTs75TwD8BAA2btyYe/uWCkT3R8v+cXkb0+e2PbwIH3emFx91S9KrBSv3tszVz5vtKcTNcpUjfOR0d3H+pgnAySWlx79HWDqZSiRQiWtTX1NtfUbsT679Es3Uk/FybuJvORqnkBEklJhTeQQh5L0AWqS/H8q8RhQAP/Vf9nZ0Ki3RAOfMUycLdm9Hp5UBKj4vxNKvYLg9hexob7aKdclhi3r45HefXoVjFwcxfDeB5zc8BABK+r7INNUnBr1ZhiDFga1aApV8TNGsWi9etmfrauVc3M7NlEBVqAgSSsypTIIQ8l8DWMMYW4m0gP9zALsC2C9hwJRy7iQKpsbMTu4UJwtWNBoWCEvUT9SHwE34xcKsHraojyveN4quwXSd8gOfXEXrornKfuJ96XR72Up2apYhrofuntLXFvZ/fBn7v/PNnBcNTQlUhYogocScyiSSfRN3OOfTAP4AQAeA3wL4b5zzr/LdbyVyOD6AN94/b+yuLtjRbu9OH2UwioJeqnbdsnlW+KJ+nC1r7J3Nj18aUmqhADOW6L4PLypRH07Zm+Kc9nZ04rV3v8A7n3YbO8g7TST6uMT74v/DdxPKfm6O37eOJ0RXDhUE0hmyr25rxUtPrlAsVjHW+ppqRKUKuckUt0oaiAxTP8jnIGq8FDIxhzrUVx6B+Mg55x8A+CCIfVUqfh6J5e700QizPeYLTNmbXroViZu/Z/ie5ZMWxxH7kJs+OCH71t0SnwBnH7k+rrM9I4or5RstCzA0NoBEMoWqCENn/7jSN1M0VZYXeJ0yZGUXzs5Hl6Djq35rIpOLgAW9PhAkFG5YmVR0Zmcp4eeR2M/Nqmdvuh1HCLq+oLqtrUlxPxy/NKQ0N44AtiiUw/EBJZwvW0SI2znJ/mdhqQsW1sWsFHtTuVuxJpDteunXZWFdDPu/80282XHBarCRbxz4bIkqhRtWHqEW8nJanfcbb+z1ZtWvkZfj6AuqLY1zlGPpbcq+tW6JMVQwqcUmmbobZTsnffz6sc9cG7FS7A/HB2yNoYXbKdv1UhYzIww3xxM4fmkI29sXzzTYYPZzJ4hSILRCXm6r84V4JHa6RtmOk03s9XouC7XWZ2IfB09eU3zsC+tivuq0m8avH/t876ji8tjzTOuMO8fF7WSiddFcfNV7B8kUt6J9aquj2PbwIsvN4rXBRjkZGUTpE1ohL8fV+SAeib3WPXE7Tj4x3/I+9mxdbblXcll4M43fVHlRPrfXd7bZCl9lE9W9HZ2KG0hmYiqJy4Nj1oTk5bdWbkYGUfqEVsjLsWymXmvbr0WnC4i+yBfUNfL69GASVT+YvmNx7GylboH09XOqnyI4HB9IW/AOKWrRCMPqRfW4Njzh+TqWo5FBlDaM89lPsty4cSM/ffp03vsplcfXIMZhytgUIiVCBrMd4433z+OdT2fCAF96coVlmec6IYjje8kA9fq+H7LFnjtmomYsbLHAKtALjOnXDAAeXdaA5Y1zrPK82Qqbmcbsdv0IIlcYY59zzjfaXg+zkJcCQd20JkERbGtrUmp5OB2jUGN56ckVlm9bqaUuhST6GYOTAAc1Ie75+eeKeIvQR6cJUs+6rK2O4olVjUoEjHwNvI6jFIwMorxwEvK8E4IqHadEFr/oSSNCdExJME7HEG4HPdEln7HorgQl6zHFsf+jLiW5J9v1EEKvJwY5ve4XPYkpyoA9z6STf55qfdBKYpJdHvte2IC25jrrM2L8+STW7Ghvzil5iCByIbQ+8mIiW1tB+epNCTnyv/Xqek4Wn58FU7d9OPnA62uqwQAIqUxy+Crb6uQ/1l8XmZR+hVBupKE/MThVKRTH0Fvi7dq0gqxqIhSQa8UnJtcB4H9hMpfjysKerwsll/MwFX/Sj29q/CB/Xl+kNJWdddq3n3MSlrjeyi4XnztBlApOrhWyyDWy3cwmi3I2HqFlS9vUdNhvXLN+Hk51vWX04k9tzXVKurtTo2f9PVO2KAA8saoRFwfG0Dsy6evcTONL8pmiYfIipdsTC2VEEmElVD5yL0Wl8t1/Nj9tKRQl8jsG03llK0Zl8sNvWdNk+e5j0YitZombf9wtW1SM72jnEAbu3FeO6SeTUj4nIF0f5a2jXXjn0268evAMXn77VMF+OwRRTEIj5EEthrnhZeEyqAXFfPA7Bie/tLyPXZtW+J6gzvaMKBOr2wRjek9MzIdOdlvjm9aqLeqZnG6YFi4FonZ7oX47BFFMQuNamY0kC68Ll6XwCO5nDG6VBeXMx2yx0scvDSmla0Wstpf0f9NiruxqEVUW9RDBXKJF5H3rUIIOUY6ERshnI5PTa8Zi2BbF3M7LT+y5XlhKT1sH3BdLnfz8iWRKSdTJth8/51tfU41436i1wCp6lx6OD3hK3yeIMBCqqJVSuOnKLWtPT/7RFzB15P6ZIvVdZD7Kf2e7LrN9HU0RM37HTBDFpiyiVkrBpVFudTT0IlTZGijI34FcR8XvdfH69BMUIlInkRyyxngk3l9W3yVRuYRmsbNUKIWolSBxymz0kqG6o73ZEvH6mmrHxUynxUW/2Y/5Ri3p39329sVl9V0SlUuoLPJSYLYtydli6fxaXL15L+sio1tikrxYKr8XRCnXIErDmr67fKozEkSpQEKeA6Xg4skF0xqDl0Qd+fOymD6xqlFxTYxNTlmFpXJJWnIjCJeW6fzD+l0ShAy5VioEpzj8bG3dZHQxBWaSifQ2aEG7oPLd32zkITgdt5BJbAQBkJCHjlyFwSnZyY9A6tvu2rQCuzevTIcicuCnx65Y2ZNBJ07lu7+gqlT6oViTB1F5kGslRJj8xIC3mGu3pCCvPn/TtnLZWJE9+dmVYUtsg3Rb5LO/YnSUKrcIJ6J0CVUceVgJKv5dj/n22nDC6zhyGaepaiHgvxHDbDDbeQhBxcqXQv4EURpQh6AiEWTii76vfLvYBDVOPdkmFo3gqdYHHRdNK4l8RbjcEtCI/CiLhKAwEuTjtaleialRQhDj9NPYQa7ZIgRdd7FUKvm6l8g9Q3iBFjsLTNDRG3ISjd8FQLeFUlMJWL8LdDvam9HSOMfWTs3rWCjCw065JaARhYFcK7NAKfg4vTZFfrPjAjoHxq3Xcmk67OU4pu5E5EIwUwq/H6I0INdKEZBvwGIv/Hl5RNdLwOZaRjZbFIxTKCC5EMxQ0hKRDRLyAhFESnmQ+Km1nm8JgmzCIzdIlscy2+GBBFEukJD7xOtjbqktUvmNFy/UWA/HB3Dgk6vpLvcM2L15pXWscqxhQxCzQV5Czhj7AYDvAhArWv+Wc/5BvoMqVfxY2cVIQMlGKTyi6w2S5VZuhZ5AaJIgypUgolb+hHO+PvNf2Yo44C/NO+gU9XKhGFEY+aTKUyQNEQbIteIDv1Z2KVjApUYxygDn6uYqtXUOgnAiCCH/A8bYSwBOA/g/OOe3TRsxxl4B8AoALF++PIDDzj7lWovcjUK4JGZ7gsvVzVVq6xwE4UTWOHLG2BEAiw1v/TGAzwDcBMAB/HsASzjnu7MdtNLiyMNKvmn7pTTh5VtHhmLbiVIg5zhyzvl2jwf4KYC/yWFsoaXUxCpoiuWSKJWngEp8AiPCSV6LnYyxJdKfvwPgfH7DCQ+VUGs614XJfGp/l9p19dtXlCCKQb5RK/+ZMXaOMfYbANsA/GEAYwoFxWhUkAu5Rl0Iq3j35pW+I2/yiUwJy3UliFIir8VOzvnvBjWQsFGKceI6ubo48vUN5+OSCMN1JYhSg8IPcyQM/tNDJ7tz8nEHEa2Ra2RKGK4rQZQaJOR5UMpx4ofjAzjRdcv6OxaNeLZui20Vl/J1JYhShIS8TDl+aciqCw4AT7U+6FkcySomiHBBQl6m6Fb1rk0rfH2erGKCCA8k5GUKWdUEUTmQkJcxZFUTRGVAPTsJgiBCDgk5QRBEyCEhJwiCCDkk5ARBECGHhJwgCCLkVHzUSrmXos1GpZ8/QZQDFS3kld7KK+jzp0mBIIpDRbtWKr1kapDnX2p1xAmikqhoIS9GR/dSIsjzr/RJkSCKSUW7Vio9jT3I8y92xUSCqGSyNl8uBNR8uTwhHzlBFJacmy8Ts09YBZFquxBEcahoH3kpQouGBEH4hYS8xKBFQ4Ig/EJCXmJUeiQNQRD+IR95iVHpkTQEQfiHhLwEoUVDgiD8QK4VgiCIkENCThAEEXJIyAmCIEIOCTlBEETIISEnCIIIOSTkBEEQIacoRbMYY0MAugPa3UIANwPa12wRxjEDNO7ZJIxjBmjchWYF59yWJVgUIQ8SxthpUzWwUiaMYwZo3LNJGMcM0LiLBblWCIIgQg4JOUEQRMgpByH/SbEHkANhHDNA455NwjhmgMZdFELvIycIgqh0ysEiJwiCqGhIyAmCIEJO6IWcMfbvGWO/YYydZYz9HWNsabHH5AXG2F7G2IXM2H/BGJtf7DF5gTH2vzLGvmKMpRhjJR2uxRj7FmOskzHWxRj7N8UejxcYYwcYY4OMsfPFHosfGGMtjLGjjLF45vfxvWKPyQuMsRrG2CnG2JeZcf9fxR5TLoTeR84Ym8c5v5P592sA2jnne4o8rKwwxv4xgP+Pcz7NGPtPAMA5/6MiDysrjLGvA0gB+K8Avs85P13kIRlhjEUBXASwA8B1AL8G8ALnPF7UgWWBMfY0gHEA73DOHy32eLzCGFsCYAnn/AxjrB7A5wCeD8H1ZgDmcs7HGWPVAD4B8D3O+WdFHpovQm+RCxHPMBdAKGYmzvnfcc6nM39+BuChYo7HK5zz33LOO4s9Dg88DqCLc36Fc54A8B6A54o8pqxwzo8BGC72OPzCOb/BOT+T+fcYgN8CWFbcUWWHpxnP/Fmd+S8UGiITeiEHAMbYf2CM9QB4EcAbxR5PDuwG8D+KPYgyYxmAHunv6wiBsJQDjLGvAdgA4GSRh+IJxliUMXYWwCCAw5zzUIxbJhRCzhg7whg7b/jvOQDgnP8x57wFwEEAf1Dc0c6QbdyZbf4YwDTSYy8JvIybIEwwxuoA/DWAf609LZcsnPMk53w90k/FjzPGQuPSEoSiZyfnfLvHTQ8C+ADAvyvgcDyTbdyMsf8NwP8M4FleQosVPq53KdMLoEX6+6HMa0SByPiY/xrAQc75fy/2ePzCOR9hjB0F8C0AoVpsDoVF7gZjbI3053MALhRrLH5gjH0LwP8J4H/hnN8r9njKkF8DWMMYW8kYiwH45wB+VeQxlS2ZRcM/B/Bbzvl/KfZ4vMIYaxIRY4yxWqQXx0OhITLlELXy1wDakI6k6Aawh3Ne8pYXY6wLwAMAbmVe+iwk0Ta/A+BPATQBGAFwlnO+s6iDcoAx9m0A/w+AKIADnPP/UNwRZYcx9i6AZ5AuqzoA4N9xzv+8qIPyAGNsM4DjAM4hfS8CwL/lnH9QvFFlhzH2PwH4S6R/IxEA/41z/sPijso/oRdygiCISif0rhWCIIhKh4ScIAgi5JCQEwRBhBwScoIgiJBDQk4QBBFySMgJgiBCDgk5QRBEyPn/AbznUq4wjioJAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.scatter(features[:, (0)].numpy(), labels.numpy(),s=10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## 读取数据集"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- 定义一个函数，能**打乱数据集中的样本顺序**并以小批量方式获取数据"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- 定义一个`data_iter`函数，该函数接收批量大小、特征矩阵和标签向量作为输入，生成大小为`batch_size`的小批量"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-07-31T02:22:16.964710Z",
     "iopub.status.busy": "2022-07-31T02:22:16.964288Z",
     "iopub.status.idle": "2022-07-31T02:22:16.970713Z",
     "shell.execute_reply": "2022-07-31T02:22:16.970081Z"
    },
    "origin_pos": 16,
    "slideshow": {
     "slide_type": "fragment"
    },
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [],
   "source": [
    "def data_iter(batch_size, features, labels):\n",
    "    num_examples = len(features)            # 获取数据集的样本数量\n",
    "    indices = list(range(num_examples))     # 生成位置列表\n",
    "    random.shuffle(indices)                 # 对数据随机排序\n",
    "    for i in range(0, num_examples, batch_size):\n",
    "        batch_indices = torch.tensor(\n",
    "            indices[i: min(i + batch_size, num_examples)])   # 记录抽取的数据位置\n",
    "        yield features[batch_indices], labels[batch_indices] # 用yield关键字生成一个iterator"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "X为tensor([[-0.4471,  0.4029],\n",
      "        [ 0.7657, -0.4713],\n",
      "        [ 0.2101, -1.3732],\n",
      "        [-1.3949,  0.5373],\n",
      "        [-0.8662, -0.8761],\n",
      "        [-0.4696,  0.0054],\n",
      "        [ 0.4594,  0.5837],\n",
      "        [ 2.7690,  0.2098],\n",
      "        [ 0.3587, -1.4239],\n",
      "        [ 0.0238, -0.8170]])\n",
      "y为tensor([[ 1.9385],\n",
      "        [ 7.3325],\n",
      "        [ 9.2775],\n",
      "        [-0.4222],\n",
      "        [ 5.4489],\n",
      "        [ 3.2386],\n",
      "        [ 3.1290],\n",
      "        [ 9.0340],\n",
      "        [ 9.7566],\n",
      "        [ 7.0458]])\n"
     ]
    }
   ],
   "source": [
    "batch_size = 10\n",
    "\n",
    "for X, y in data_iter(batch_size, features, labels):\n",
    "    print(f'X为{X}\\ny为{y}')\n",
    "    break"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## 初始化模型参数"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- 通过从均值为0、标准差为0.01的正态分布中采样随机数来初始化权重\n",
    "- 将偏置初始化为0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-07-31T02:22:16.974692Z",
     "iopub.status.busy": "2022-07-31T02:22:16.974223Z",
     "iopub.status.idle": "2022-07-31T02:22:16.978612Z",
     "shell.execute_reply": "2022-07-31T02:22:16.977979Z"
    },
    "origin_pos": 19,
    "slideshow": {
     "slide_type": "fragment"
    },
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [],
   "source": [
    "w = torch.normal(0, 0.01, size=(2,1), requires_grad=True)  # 记录偏导数\n",
    "b = torch.zeros(1, requires_grad=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## 定义模型"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- 将模型的输入和参数同模型的输出关联起来"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-07-31T02:22:16.982895Z",
     "iopub.status.busy": "2022-07-31T02:22:16.982343Z",
     "iopub.status.idle": "2022-07-31T02:22:16.985872Z",
     "shell.execute_reply": "2022-07-31T02:22:16.985262Z"
    },
    "origin_pos": 22,
    "slideshow": {
     "slide_type": "fragment"
    },
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [],
   "source": [
    "def linreg(X, w, b):  # net\n",
    "    \"\"\"线性回归模型\"\"\"\n",
    "    return torch.matmul(X, w) + b"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## 定义损失函数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-07-31T02:22:16.989920Z",
     "iopub.status.busy": "2022-07-31T02:22:16.989411Z",
     "iopub.status.idle": "2022-07-31T02:22:16.993061Z",
     "shell.execute_reply": "2022-07-31T02:22:16.992350Z"
    },
    "origin_pos": 24,
    "slideshow": {
     "slide_type": "fragment"
    },
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [],
   "source": [
    "def squared_loss(y_hat, y):\n",
    "    \"\"\"均方损失\n",
    "    y_hat：预测值\n",
    "    y：真值\n",
    "    \"\"\"   \n",
    "    return (y_hat - y.reshape(y_hat.shape)) ** 2 / 2  # 将y转换为与y_hat相同的形状"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## 定义优化算法"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- 实现小批量随机梯度下降算法"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-07-31T02:22:16.997231Z",
     "iopub.status.busy": "2022-07-31T02:22:16.996703Z",
     "iopub.status.idle": "2022-07-31T02:22:17.001042Z",
     "shell.execute_reply": "2022-07-31T02:22:17.000440Z"
    },
    "origin_pos": 27,
    "slideshow": {
     "slide_type": "fragment"
    },
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [],
   "source": [
    "def sgd(params, lr, batch_size):  # params: [w,b]\n",
    "    \"\"\"小批量随机梯度下降\n",
    "    lr：学习速率\n",
    "    \"\"\"\n",
    "    with torch.no_grad(): # 在该上下文管理器下，所有计算得出的tensor的requires_grad都自动设置为False。\n",
    "        for param in params:\n",
    "            param -= lr * param.grad / batch_size  # 更新参数，学习速率lr控制更新大小，批量大小batch_size规范化步长\n",
    "            param.grad.zero_() # 为什么要将梯度置零？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "```python\n",
    "torch.no_grad\n",
    "```\n",
    "- 上下文管理器（context manager），使梯度计算失效。当不需要使用`Tensor.backward()`的时候，使梯度计算失效可以节省计算梯度占用的内存\n",
    "- 即使输入的`requires_grad=True`，利用该上下文管理器后，计算出来的`Tensor`的`requires_grad=False`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## 正式开始训练"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- 训练的流程\n",
    "    * 初始化参数\n",
    "    * 重复以下训练，直到完成\n",
    "        * 读取批量数据，获得预测，计算损失\n",
    "        * 计算梯度$\\mathbf{g} \\leftarrow \\partial_{(\\boldsymbol{w},b)} \\frac{1}{|\\mathcal{B}|} \\sum_{i \\in \\mathcal{B}} l(\\mathbf{x}^{(i)}, y^{(i)}, \\boldsymbol{w}, b)$\n",
    "        * 更新参数$(\\boldsymbol{w}, b) \\leftarrow (\\boldsymbol{w}, b) - \\eta \\mathbf{g}$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "\\begin{definition}\\label{def:epoch}\n",
    "**迭代周期**（epoch）：使用训练集的全部数据对模型进行一次完整训练，也被称为**一代训练**\n",
    "\\end{definition}\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- 在每个迭代周期（epoch）中，使用data_iter函数遍历整个数据集， 并将训练数据集中所有样本都使用一次（假设样本数能够被批量大小整除）"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "- 训练流程\n",
    "\n",
    "<center><img src=\"../img/3_linear_network/epochMiniBatchTraining.svg\" width=70%></center>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-07-31T02:22:17.011859Z",
     "iopub.status.busy": "2022-07-31T02:22:17.011351Z",
     "iopub.status.idle": "2022-07-31T02:22:17.128087Z",
     "shell.execute_reply": "2022-07-31T02:22:17.127385Z"
    },
    "origin_pos": 32,
    "slideshow": {
     "slide_type": "slide"
    },
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "epoch 1, loss 0.009596\n",
      "epoch 2, loss 0.001577\n",
      "epoch 3, loss 0.000296\n",
      "epoch 4, loss 0.000089\n",
      "epoch 5, loss 0.000057\n"
     ]
    }
   ],
   "source": [
    "lr = 0.01  # 超参\n",
    "num_epochs = 5  # 超参\n",
    "net = linreg    # 实例回归模型\n",
    "loss = squared_loss # 实例损失函数\n",
    "\n",
    "for epoch in range(num_epochs):\n",
    "    for X, y in data_iter(batch_size, features, labels):    # 遍历全部数据集\n",
    "        l = loss(net(X, w, b), y)                           # net(X,w,b)预测，loss(net,y)计算损失\n",
    "        l.sum().backward()                                  # 反向传播计算参数的梯度（偏导数）\n",
    "        sgd([w, b], lr, batch_size)                         # 更新参数\n",
    "    with torch.no_grad(): \n",
    "        train_l = loss(net(features, w, b), labels)         # 一个epoch输出一次优化的参数与损失\n",
    "        print(f'epoch {epoch + 1}, loss {float(train_l.mean()):f}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "比较真实参数和通过训练学到的参数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-07-31T02:22:17.132400Z",
     "iopub.status.busy": "2022-07-31T02:22:17.131912Z",
     "iopub.status.idle": "2022-07-31T02:22:17.137249Z",
     "shell.execute_reply": "2022-07-31T02:22:17.136497Z"
    },
    "origin_pos": 35,
    "slideshow": {
     "slide_type": "fragment"
    },
    "tab": [
     "pytorch"
    ]
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "w的估计误差: tensor([[ 0.0015],\n",
      "        [-0.0029]], grad_fn=<SubBackward0>)\n",
      "b的估计误差: tensor([0.0016], grad_fn=<RsubBackward1>)\n"
     ]
    }
   ],
   "source": [
    "print(f'w的估计误差: {true_w - w.reshape(true_w.shape)}')\n",
    "print(f'b的估计误差: {true_b - b}')"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "幻灯片",
  "hide_input": false,
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  },
  "latex_envs": {
   "LaTeX_envs_menu_present": true,
   "autoclose": true,
   "autocomplete": true,
   "bibliofile": "biblio.bib",
   "cite_by": "apalike",
   "current_citInitial": 1,
   "eqLabelWithNumbers": true,
   "eqNumInitial": 1,
   "hotkeys": {
    "equation": "Ctrl-E",
    "itemize": "Ctrl-I"
   },
   "labels_anchors": false,
   "latex_user_defs": false,
   "report_style_numbering": false,
   "user_envs_cfg": false
  },
  "rise": {
   "autolaunch": false,
   "enable_chalkboard": true,
   "scroll": true
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  },
  "vscode": {
   "interpreter": {
    "hash": "34418ffb6f02e522390adb0e13441cc75f901cd11cccb4f6f613643b4b4d2a0b"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
